Choosing Workgroup Members 1 Running head: CHOOSING WORKGROUP MEMBERS Choosing Workgroup Members: Balancing Similarity, Competence, and Familiarity

This study explores one of the contributors to group composition – the basis on which people choose others with whom they want to work. We use a combined model to explore individual attributes, relational attributes, and previous structural ties as determinants of work partner choice. Four years of data from participants in 33 small project groups were collected, some of which reflects individual participant characteristics and some which is social network data measuring the previous relationship between two participants. Our results suggest that when selecting future group members people are biased toward others of the same race, others who have a reputation for being competent and hard working, and others with whom they have developed strong working relationships in the past. These results suggest that people strive for predictability when choosing future work group members. Choosing Workgroup Members 1 Choosing Workgroup Members: Balancing Similarity, Competence, and Familiarity People often play either a direct or an indirect role in choosing their work partners. In volunteer organizations, people decide which group or committee to join. Academics decide who to collaborate with on research projects. And, in business organizations, an increasing number of recruiting processes are partially composed of evaluations by prospective peers. In a recent review of the diversity literature Williams and O’Reilly (1998) observe that an important unanswered question in diversity research concerns what drives the composition of groups. Given the importance of group composition on group structure, performance, and satisfaction (see Moreland & Levine, 1994 for a review), understanding the way that people choose their group members is a crucial step in understanding what leads to the creation of successful groups. In this study, we explore one of the antecedents to group composition – the basis on which people choose group members. Although preferences for working with others may evolve over time, there is often a point at which a person must make an explicit choice. By focusing on this explicit choice, we hope to provide valuable insight into how people use previous experience as input to the composition of future work groups. In this paper, we have used social psychological and sociological theories predicting interaction choice as well as social network theories to investigate how people choose work group members. To date, much of the research on interaction choice has focused on social relationships such as friendship (e.g. Festinger, 1954; Lazarsfeld & Merton, 1954; Carley & Krackhardt, 1996) and has neglected instrumental relationships such as work ties. Although Choosing Workgroup Members 2 friendships and informal ties have been identified as important relations in organizations (e.g. Brass & Burkhardt, 1992; Krackhardt, 1992), formal work relationships also are critical to the accomplishment of organizational goals. We explore how individual attributes, relational attributes, and previous network ties affect the choice of future group members. We look at stated preferences for future group members rather than inferring choice based on existing group composition or changes in interaction within existing groups. This approach isolates the choice process by excluding other social and organizational pressures that influence the composition of teams. Theory Organizations are deliberately constructed to achieve specific goals (Etzioni, 1964). But, rampant uncertainty in organizational life often threatens the achievement of the organization's objectives (Thompson, 1967). A prevalent theme in organizational research is the reduction of uncertainty as a way of regaining control over the fate of the organization (Thompson, 1967). Kanter observed that this need to limit uncertainty results in a drive to control and limit participation in both work and friendship networks (1977, chapter 3). We suggest that this need to gain control by reducing uncertainty also is operating at the individual level. Thus, in putting together project teams and work groups, individuals will, when given the chance, construct the group that they feel is most likely to succeed. Further, they will do this by choosing as co-group members those whom they feel have a higher probability of producing a positive pay-off. This suggests that workers will decide who they want to work with in the future based on those characteristics that create the most confidence in the other's ability to be a “good” group member. Such a process can be seen as one of social exchange Choosing Workgroup Members 3 in which partners conduct (consciously or unconsciously) a cost-benefit analysis weighting the potential contributions of the prospective group member as well as the potential costs (e.g. Blau, 1964; Thibaut & Kelley, 1959). But, what are these characteristics or indicators that could increase or decrease one's confidence in another's ability to contribute to a group, and hence increase the likelihood that the group will succeed? Theories on friendship, group formation, and network evolution all suggest ways in which confidence in a positive work relationship might be achieved. Various scholars have argued that similarity and affect (e.g. Lazarsfeld & Merton, 1954; Lott & Lott, 1964; Carley, 1991; Carley, 1999), competence (e.g. Thibaut & Kelley, 1959), and familiarity (e.g. Zander & Havelin, 1960) increases people's preference to interact and work with one another. Others have argued for the importance of individual indicators of competence such as status (Berger, Wagner & Zelditch, 1985) and values congruence (Cable & Judge, 1996). In this paper, we attempt to explore concurrently the impact of individual attributes and previous work relationships on the choice of future group members. We consider the relative contributions of three uncertainty reduction mechanisms — homophily (similarity), reputation for competence, and familiarity — in predicting group member choice. Each of these factors has been extensively explored in the literature. But, most of this work examines a single factor at a time (homophily, familiarity, etc.). In the study reported here, we build on and integrate existing research by simultaneously tapping all three factors and using multiple indicators for each of these factors. This allows us to examine the relative contributions of each factor and build an integrated model depicting the basis on which work group members are chosen. Choosing Workgroup Members 4 Homophily Lazarsfeld & Merton (1954) coined the term “homophily” to refer to a tendency for people to be attracted to others who have similar attitudes, beliefs, and personal characteristics. Similar people want to interact with one another, join the same groups, and tend to value each others' contribution more than contributions from dissimilar people (see Lincoln & Miller, 1979; Miller and Smith-Lovin, 1987; Tsui & O'Reilly, 1989). In fact, the relationship between homophily and affect is one of the most consistent findings in interpersonal research (see Lott & Lott, 1965 for a review). One of the explanations for this phenomenon is that people with similar demographic backgrounds have more “language compatibility” than do those who are dissimilar (March & Simon, 1958: 67). Others have argued that common knowledge or shared mental models may produce more opportunities for interaction (Carley, 1990; Carley, 1991; Kaufer & Carley, 1993). In their review of the diversity literature, Williams and O’Reilly (1998) observe that similarity can result in more frequent communication, more positive attitudes, and decreased turnover in organizations. Further, demographic homophily fosters trust and reciprocity and enhances instrumental relations (Lincoln & Miller, 1979). For a variety of reasons, homophily increases the ease of communication and improves the predictability of behaviors and values. Therefore, when choosing group members, we expect people to favor peers who are similar to themselves as a means of increasing predictability within the group. H1: People will choose to work with others who are similar to themselves. Choosing Workgroup Members 5 Project groups however do not just come together for social reasons. They come together to do a particular task. To successfully complete a task the group will undoubtedly need a particular set of skills. As individuals work together they become aware of both the similar and the complementary skills of other group members. Groups with complementary skills can cover a wider range of possible problems and may bring a host of alternative information thus increasing the overall knowledge and information processing capability of the group which should in turn lead to higher group performance (Hollenbeck, Ilgen, Sego, Hedlund, Major & Phillips, 1995; see also Jehn, Northcraft, & Neale, 1997; Pelled, Eisenhardt, & Xin, 1997). Groups with complementary skills should therefore be more successful to the extent that their skills are necessary to contribute to the task. To increase the likelihood that the group will succeed and the probability that a potential group member will provide value to the team, people should use skill complementarity as a factor in choosing work group members. In other words, individuals may trade some certainty in communicative and individual performance predictability for certainty in skill coverage. Thus, in terms of choosing future group members there might be both a general homophily effect (i.e. for demographic factors) and a contradictory complementarity effect for skills. H2: People will choose to work with others who have skills complementing their own. Reputation for Competence Choosing Workgroup Members 6 Choosing a member of a work group can be viewed as an organizational decision like any other resource allocation or technical decision. Within the context of resource allocation, Pfeffer, Salancik, & Leblebici (1976: 228) introduced the universalism-particularism dimension of organizational decision making. Particularistic criteria are those that depend on the relationships between organizational members (e.g. homophily). Universalistic criteria are independent of the relationship between people. For example, individual competence to perform a job is a universalistic criterion. In instrumental groups, we predict that use of particularistic criteria will be complemented by universalistic criteria in the service of group goals. As another means of reducing uncertainly, we expect people will use a person's ability to contribute to the task as a criterion when choosing group members. Some research supports the idea that people will choose others to work with based on the extent to which they can be expected to contribute to the group goal or task (Ridgeway, 1984; Thibaut & Kelley, 1959). Other research suggests that people may fall back on more particularistic indicators such as familiarity because competence is too difficult to judge (Perrow, 1972: 11). Although assessment is difficult, we believe that people may not be willing to forgo this vital information about individual competence. Instead, they will gather information on individuals’ competence through their interaction with others in the organization. Reputations then form as people search for indicators as to others' abilities and signal their own competence by alerting others to organizational accomplishments (Kilduff & Krackhardt, 1994; Spence, 1973). We argue that reputational information about the abilities and work habits of people filters into the organization and generates reputations for competence. Although performance reviews, examinations, Choosing Workgroup Members 7 peer evaluations and such are deemed “private” information, we believe that enough of this information seeps into the organization to create relatively accurate reputations about the competence of organizational members. One indication of the competence of potential group members is whether or not they have the skills required for successful completion of the project. For instance, in a project group chartered to write a computer program, the ability to program may be highly desirable. Although objective measures of individual capabilities are generally not shared and are difficult to judge, we argue that people rely on relevant information that flows over the organizational grapevine and thus make choices based on the relevant skills of their peers in the organization. H3 People will choose to work with others who have instrumental skills that are relevant to the group task. Familiarity Another method of reducing uncertainty is to work with those whose personalities and work practices are familiar. Extensive research demonstrates that people who interact with one another are more likely to be positively disposed toward one another (e.g. Byrne, 1961; Festinger, 1953; Zander & Havelin, 1960). Even groups of strangers express a preference to work with one another once they interact (Zander & Havelin, 1960). In work situations, there are several ways in which familiarity reduces uncertainty. First, there is an expectation that the person's behavior will be similar in future interactions regardless of whether the prior interaction was positive or negative (see Rhodes, Newman, & Ruble, 1990). Second, each time new Choosing Workgroup Members 8 members are added to a group, those members need to be “brought up to speed” and socialized as to the groups' work processes. When unfamiliar members are added to a group, coordination losses are incurred (Goodman & Leyden, 1991). Without any personal experience with another individual, it is difficult to anticipate the extent to which new work practices will need to be negotiated and therefore the magnitude of the coordination losses. Because of the increased predictability in behavior and coordination costs, we expect people who have worked together to choose to work together in subsequent groups. Of course, there are many cases where people have had negative experiences with their group members. Because the objective is to maximize the likelihood that the group will achieve its instrumental goals, it follows that people will only choose to work with peers with whom their previous relationship has been positive. In fact, previous research found that people in more successful groups prefer to work with others in that group more often than do those in less successful groups (Zander & Havelin, 1960). H4 People will choose to work with others with whom they have worked with in the past, particularly if the relationship resulted in a successful outcome. Method Overview Data come from 33 software development groups with 3 to 7 members collected over four years. Group members were undergraduates in an information systems (IS) major at Carnegie Mellon University. In their junior year, they were required to work on an assigned group for 4 months Choosing Workgroup Members 9 with the goal of developing a computer-based information system (project 1). During their senior year, all students were required to work on a different project with a different group (project 2). The purposes of the group projects in the curriculum are to help students learn to apply their knowledge and skills and to provide conditions for actively acquiring teamwork skills. The junior project groups were ongoing, bounded groups with shared responsibility for their group performance. Group success was evaluated primarily by comprehensive grading criteria for the overall group project. In addition, group members' individual grades were adjusted based on confidential peer evaluations completed by group members at the end of the term. Groups in the junior year were disbanded and assigned to new groups in the senior year. At the end of the junior year, students were asked to nominate peers with whom they would like to work on project 2 in their senior year. These choices were important because the instructors used them to assign students to groups for project 2. We study the determinants of these choices for who one wants to work with in project 2. Data Collection Procedure We collected data from four main sources. (1) Surveys of perceptions of group process . Group members individually and privately completed three surveys of group behavior during project 1 at 1, 2, and 3 months into the project. (2) Weekly time reports . On a weekly basis, group members reported the hours they spent on project 1. Time reports had to be signed by the group manager to verify accuracy of the reports. (3) External sources. These included demographic information, overall grade point average, and grades in other relevant courses. These data were taken from student records. (4). Choosing Workgroup Members 10 Peer nomination survey . Four months following the completion of project 1, students were asked to nominate up to five peers with whom they wanted to work on project 2 the following semester. Students were assured that nominations were confidential and would be used as input to form groups for the following term. Measures The data collected in this study are predominantly dyadic; that is, they reflect a relationship between two individuals. Dyadic data allows us to directly examine who chooses whom. It also allows us to explore how the nature of previous relationships leads to future relationships. To examine choice, dyadic matrices were created for most of our independent variables and for the dependent variable in the study. The matrices reflect i's relationship to j in all cases Dependent variable: group member choice matrix. Each student in the program filled out a survey during the Spring semester (approximately 4 months after the completion of project 1). In this survey, they were asked to name up to five of their peers that they wanted to work with on project 2 another four months hence. Each student made this choice at only one point in time. Students were assured that their choices would be honored to the extent possible given the goals of the course. These data were then coded into a choice matrix which represented whether or not person i indicated that he wanted to work with person j in the subsequent year. Each row in the dependent variable matrix therefore represented those people chosen by person i with a “1” and those people not chosen by person i with a “0”. Similarly, each column in the matrix reflected who specifically requested to Choosing Workgroup Members 11 work with person j and who did not identify person j as a preferred group member. Independent variables: homophily matrices. Previous studies exploring homophily and attraction have examined similarity in gender, race, ethnicity, tenure, age, education, and a variety of other demographic measures. In our setting, the subjects all have the same tenure in the organization (college juniors), are approximately the same age, and have essentially the same educational background. According to McPherson and Smith-Lovin (1987), our groups have a high level of induced homophily — that is, their options of interacting with dissimilar people are limited because of the built-in homophily within the organization. We therefore focused on gender, race, and role in our homophily measures. Data on gender and race were collected from student records. Two homophily matrices were then created in which a “1” indicates that the two subjects are the same on that dimension and a “0” indicates that they are different. Therefore, both rows and columns in the “same gender” matrix have “1's” for every dyad of the same gender. In the race data, subjects were identified as either Black, Asian, or White. The rows and columns in the “same race” matrix reflect whether or not the two people are the same race. Similarly, a dichotomous (0, 1) matrix reflects whether or not i and j performed the same role in the group during project 1. Possible roles included manager, programmer, analyst, and documentor (technical writer). Participants were categorized based on the function in which they spent a majority of their time on project 1. Those pairs who performed the same role in project 1 received a “1” in the cell corresponding to their relationship and those who did not perform the same role received a “0”. Choosing Workgroup Members 12 Independent variables: reputation for competence. There are several measures of competence that we hypothesize have become shared information in the organization and have become reputational. We selected five measures that we believe most closely map to performance evaluations in other types of organizations — grade point average, two relevant course grades, effort, and project success. In most traditional organizations, the boss evaluates the contribution of each of his subordinates. In a classroom setting, this role is played out by the instructor. The instructor determines the extent to which the student's performance meets the standards set forth by the organization. Therefore, we choose overall grade point average and individual course grades in an analysis and in a database design course as external evaluations of competence. Grade point average (GPA) of j should be in indication of j's overall intelligence and ability to perform well in a college setting. The analysis course grade refers to the individual score on exams and homework in a course covering the analysis and design of software systems. The database course grade refers to the individual's performance in a database design and development course. Skill in analysis, database design, and database development are critical for successful performance on project 1 and project 2. Again, we are hypothesizing that high performance in these courses becomes socially shared information as people signal their own value and search for indications of others' competence. GPA, analysis course grade, and database course grade are treated as attributes (columns) of person j. GPA is a continuous variable with 4 representing an A average and 0 representing a failing average. Analysis course grade and database course grade are 5-level variables with 4 representing the highest grade (A) and 0 representing failure. Choosing Workgroup Members 13 We also included a measure for how successful person j’s group was for project 1. People who are members of successful groups may be seen as “basking-in-reflected-glory” and therefore be viewed positively regardless of their contribution to the group (Cialdini et al., 1976). Groups were ranked against one another based on their performance on the project 1. The rankings were generated by the instructor for the project course with input from clients for whom the projects were being developed. This variable was constructed as a rank variable in which higher values represented higher levels of success. Finally, we included a measure for the amount of effort people put into project 1. Whereas competence is a critical indicator of performance capacity, effort is an indicator of the individual's willingness to apply that capacity. Further, the number of hours put in by a peer may be easier for observers to evaluate than skill level. An attribute measure was created by totaling the number of hours reported by j on the weekly time reports for project 1. Independent variables: familiarity matrices. Familiarity is a structural variable reflecting the extent to which the structure of project 1 threw two people together. Historically, familiarity has been operationalized in a variety of ways including opportunity to interact based on physical proximity (e.g. Festinger, 1953), shared knowledge about a work process (e.g. Goodman & Leyden, 1991), and participation in the same group (e.g. Zander & Havelin, 1960). In their review, Lott and Lott (1965) conclude that familiarity requires opportunity to observe behavior and actual interaction. In this study, we have two measures of structural familiarity —participation in the same group and amount of interaction during project 1 at the end of month 1, month 2, and month 3. We also have a measure for group success as an indicator of the quality of the interaction. Choosing Workgroup Members 14 People who were in the same group in project 1 were assumed to be familiar with each other. Project 1 was a four month long effort in which groups worked together extensively. Therefore, being in the same group meets Lott and Lott's criteria for being able to observe behavior. For the “same group” measure, a matrix was created that indicates whether or not two students occupied the same group in project 1. If they belonged to the same group, the cell representing the relationship was filled in with a “1” and, if they belonged to different groups, it was filled in with a “0.” We also used direct measures of interaction. Students were asked, at three points during project 1, to report for each other student in their group the extent to which the respondent was working with that other group member. Students rated their interaction with each other member of their group using a scale of 0 to 5 with 0=did not interact and 5=worked with a lot. Matrices were then created for each of the three time periods. For example, if the cell WorkedWith ij(t) = 5, that means that i perceives that he works with j at level 5. Our hypothesis (H4) suggests that familiar people will only want to interact again if the previous interaction was rewarding. To test this, we created an interaction term in which we multiplied the “same group” variable above with our measure of project success. From these data, an interaction matrix based on the group's overall success was created. If the two people were in the same group, their project rank occupied the cell for the dyad. Analysis In our analysis, we explored three nested models. The order of the models reflects the prominence of that category in the literature and the expected Choosing Workgroup Members 15 strength of each predictor within the model. We added the structural factors last to examine the additional contribution these factors provide to understanding team member choice. First, we examined the contribution of homophily to peoples’ choice of group members. Homophily was selected first because it is one of the most robust findings in social psychology and should therefore have strong explanatory effects as a predictor (Lott & Lott, 1965; McPherson & Smith-Lovin, 1987). Second, we examined the effect of competence. Competence was selected second because theory predicts that competence should be a strong individual level predictor of team member choice for instrumental teams. Finally, we examined familiarity. Familiarity is the third category added to the model because we use network techniques to explore not only the familiarity between two people, but their relative familiarity within the larger organization. By looking at these relational factors, we hope to add both a better understanding of the category of familiarity as well as contribute a new analysis technique to the question of how people choose work group members. As is common in performing network analysis, the dependent variable of interest to us is dyadic in form. The number of observations, then, is considerably larger than the number of people in the study; in the current case, the N people in each year creates N(N-1) dyadic observations for that year. It has been long recognized that traditional statistical methods cannot be applied to such data because these N(N-1) observations are not independent of one another (Proctor, 1975; Laumann and Pappi, 1976; Schott, 1987). For example, in the current research, the interaction effect (reflecting the extent to which i worked with j in project 1), includes a value for i's level of interaction with every other person in the class. But, some people are more gregarious than others and may have higher scores, in general, than Choosing Workgroup Members 16 their peers. This systematic interdependence occurs because each of the row values is taken from the same source. Such row (or column) independence can bias OLS tests of significance (see Krackhardt, 1988). Indeed, even robust nonparametric tests such as bootstrapping and jackknifing are sensitive to this assumption of independence of observations. Over the past decade, several attempts have been made to address this problem by modeling the type of dependence among observations that might occur in a given context (e.g., Mizruchi and Galaskiewicz, 1994; Wasserman and Faust, 1994; Lincoln, 1984, Kenny et al., 1996). These approaches work well if in fact the data conform to the model specifications. However, the number of parameters that could reasonably influence the dependence among these observations is far greater than the number of observations (Krackhardt, 1988), and thus no assurances can be made that the model is correctly specified. The QAP Test An alternative approach, however, was developed for bivariate cases by Larry Hubert (1987) and extended to multivariate cases by Krackhardt (1987; 1988). This approach, called QAP (quadratic assignment procedure), has been shown to be robust against a wide array of sources of autocorrelation frequently assumed in dyadic network problems. For example, Krackhardt (1988) demonstrated that traditional tests of significance will have a probability of greater than .5 of finding significant results in a sample drawn from a population where the null hypothesis is true even when the dyadic data are only moderately autocorrelated. He further demonstrated that QAP tests are robust against even extreme autocorrelation in the data. That is, even in these extreme cases, the multivariate QAP tests found significant Choosing Workgroup Members 17 results exactly alpha portion of the time which is exactly what one expects and demands of an unbiased significance test. The QAP test is a simple two-step procedure. First, the data, originally in matrix form, are "vectorized" (strung out into a vector of observations) of length N(N-1) and an ordinary multiple regression is performed on them, resulting in a traditional beta coefficient estimated for each independent variable. These coefficients are unbiased even if the data are autocorrelated (Judge, et al, 1990, p. 27). The problem, however, is that the traditional estimates of standard errors of these coefficients is very sensitive to autocorrelation in the data hence they perform poorly as the basis of significance tests of these coefficients. The second step in the QAP test, then, is to generate a null hypothesis reference distribution against which the observed coefficient can be compared to determine its statistical significance. This reference distribution is created by randomly permuting the dependent variable matrix multiple times, each time recalculating the regression coefficients for each independent variable predicting the permuted dependent variable. In our case, with 999 permutations, this would result in a set of 999 beta coefficients for each independent variable against which the observed beta could be compared. If the observed beta is larger than all 999 of the betas generated under the null hypothesis (represented by the randomly permuted data), then we say that that beta is significantly different from random at the .001 level (the exact pvalue is calculated as (S+1)/(K+1), where S is the number of coefficients based on randomly permuted versions of the dependent variable that are greater than or equal to the observed coefficient and K is the number of permutations performed). Choosing Workgroup Members 18 The key to the QAP test is the exact nature of the permutation. The permutation is of a restricted form: All rows and columns of the matrix are permuted identically. That is, if row 3 and row 7 are switched, then columns 3 and 7 are switched in the same manner. This form of permutation is tantamount to a "relabeling" of the matrix (actors switch places) while the structure of the matrix remains unchanged under each permutation. Thus, to the extent there is autocorrelation in the data, the autocorrelation is preserved under each permutation (as is the extent of collinearity among all independent variables). This leads to a conditional permutation test, where the null hypothesis reference distribution is based only on cases where the data are constrained by the same structural (row, column and reciprocity) features as the observed data. It is the nature of this permutation that permits the test to be so robust against the kind of autocorrelation that is common in network data. QAP was first suggested for the bivariate case by Hubert and others (Hubert & Shultz 1976, Backer & Hubert, 1981, Hubert & Golledge 1981), and then adapted to test for spuriousness (Krackhardt, 1987) and to the case of multiple regression (Krackhardt, 1988). MRQAP (Krackhardt, 1988; 1993) is a straightforward extension of Hubert's (1987) bivariate permutation test. Hubert's test is appropriate for any problem that can be formulated as a correlation between two NxN Cartesian product matrices. Network data, such as we are using, are of this form. Krackhardt (1992) extended Hubert's procedure by adding pairwise reciprocity. The permutation version of MRQAP (Krackhardt, 1993), which we use in this paper, has several advantages over the analytic MRQAP procedure (Krackhardt, 1988). Among the advantages of the permutation approach are that it is somewhat less Choosing Workgroup Members 19 biased when there are many independent variables and it permits testing the overall R-square and intercept against a random null hypothesis. Krackhardt (1993) suggests the use of the Multiple Regression Quadratic Assignment Procedure (MRQAP) to deal with this problem of biased significance tests. In the MRQAP analysis, the dependent variable is an N x N matrix rather than a vector as is normally the case in regression analysis. The first step in the MRQAP procedure is to calculate the OLS estimates of the regression coefficients based on the N(N-1) observations contained in the matrices (the diagonals are ignored). Then the rows and columns of the dependent variable matrix are permuted to give a new “random” matrix. The OLS regression is then repeated with this new permuted matrix, resulting in different beta coefficients. The procedure is repeated... resulting in a distribution of beta values based on these permuted matrices. This distribution of values becomes the reference distribution against which the observed coefficients are compared. If less than 5% of the betas for a given variable derived from the permuted regressions are larger than or equal to the observed beta for that variable, we say that the beta is significant at the .05 level (one-tailed). If less than 1% of the betas are larger than or equal to the observed beta, we say that the beta is significant at the .01 level. The permutation based MRQAP procedure is illustrated in Carley and Krackhardt (1996). Since MRQAP is an OLS procedure the first moments (the beta estimates) are unbiased estimates. Further, both the analytic and the permutation version of MRQAP are superior to OLS in testing the null hypothesis (beta=0) for network data. In other words, such hypotheses tests are robust against independence of observation problems one encounters in network data (Krackhardt, 1988). As noted by Carley and Krackhardt (1996) “an OLS test of regression coefficients for random network data drawn from a Choosing Workgroup Members 20 null population will reach the .05 level of significance more than half the time. In contrast, the MRQAP test of these same coefficients reaches the .05 level of significance about 5% of the time, as it should, independent of the extremity of the network autocorrelation in the data.” The Meta Analysis Meta analysis allows one to combine the results from independent studies (samples) to make an overarching statement about the strength and significance of a hypothesized relationship. Meta analysis is commonly seen in reviews of an area where many published studies often use different measures and models but attempt to answer a similar question. The application of meta analysis in these cases has come under severe criticism because of many factors, such as the fact that dissimilar models are inappropriately combined and that relying on published studies underrepresents those research efforts that found no results (and thus were never published). The current study, however, is a good example of how meta analysis can be usefully applied (Hedges and Olkin, 1985). Identical models and measures are used in each sample, and there is no selective sampling based on the results. However, the interpretation of a meta analytic result (both for the effect size and significance of the result) depends on the assumption that the samples (studies) are drawn from the same population, vis-=E0-vis the variables under consideration. Hedges and Olkin (1992) recommend performing a Q-test across the samples or studies to determine whether they differ significantly on the variables of interest for the meta analysis. If the samples are not significantly different from each other on the Q-test, then we can proceed with a meta-analytic test to see if the combined results are Choosing Workgroup Members 21 significant (as well as calculate a combined effect size). If, on the other hand, the Q-test indicates that the samples do differ significantly from each other, then it would be inappropriate to suggest that results of the meta-analysis indicate anything about a combined result (as opposed to merely different laws governing each sample separately). According to Hedges and Olkin, the Q-test must be precede any metaanalytic test. Of the four overall meta-analytic significance tests reviewed by Hedges and Olkin (1985), the Stouffer test was one that encountered the fewest difficulties and restrictions in interpretation and application. As such, they recommend the Stouffer test and we follow their recommendation in this paper. However, performing a Q-test across our four samples poses a serious problem. The Q-test relies on estimates of the standard errors of the beta coefficients being tested. As noted earlier, traditional estimates of standard errors in dyadic data cases has the potential of being severely biased. Thus, we cannot employ a traditional Q-test here. Faced with a similar dilemma, Krackhardt and Kilduff (1999) developed a Q-test for dyadic data using the same logic for QAP tests: They calculate pseudo standard errors of the coefficients based on the random permutations of the data. Each of these estimates of standard errors, then, is conditioned on the structure and autocorrelation in the data, just as were the direct significance tests. We will use the same procedure Krackhardt and Kilduff developed here. Specifically, a Q-test is a three-step procedure. First, the pseudo-standard errors of each beta coefficient is calculated. Second, a weighted combined coefficient (effect size) is calculated, taking into account its pseudo standard errors across the sites. Third, a Q-test is performed to assess whether or not Choosing Workgroup Members 22 the sites could reasonably be expected to be drawn from the same population, given the standard errors estimated in step one. The calculation of the standard error of the beta coefficient was straightforward. For each given independent variable, a vector of beta coefficients of length 999 was created from the regression calculations generated from the 999 permutations of the dependent variable under the null hypothesis. The standard deviation of this vector of randomly generated beta coefficients was used as the estimate of the standard error for that beta. The weighted combined coefficient, termed "betaplus" by Hedges and Olkin (1985), is calculated as the weighted average of the coefficients observed across the four sites. The weight of each of the four site coefficients was the inverse of the square of the standard error of estimate for that beta at that site. This weighting scheme gives priority to beta estimates with a smaller standard error. The Q for each independent variable is calculated by summing the weighted squared differences between each site's beta estimate and the combined estimate, betaplus (see Krackhardt and Kilduff's, 1999, article in the Journal of Personality and Social Psychology for a detailed example of these calculations). The weight for each term in this sum of squares is again the inverse of the square of the pseudo standard error for that beta at that site. This Q-value is distributed as a chi-square under the null hypothesis that all sites are drawn from the same population. The degrees of freedom for this Q test is the number of terms in the calculation of Q minus one (in the present case, that equals 4 sites -1=3D3 degrees of freedom). The Data The data in this study are gathered from 4 distinct groups of students over 4 years. Each year was analyzed separately and a meta-analysis performed to Choosing Workgroup Members 23 integrate the findings into a single measure (see Wolf, 1986). We used Stouffer's combined (Stouffer, Suchman, DeVinney, Star & Williams, 1949) because of its straightforward calculations, its sensitivity to results in opposite directions, and because it has already been successfully applied to the metaanalysis of MRQAP results (see Krackhardt & Kilduff, 1999). In Stouffer's test, p-values from the regression analysis are converted into z-scores and summed over the tests performed. This sum is then divided by the square root of N, where N equals the number of tests. We have reported the zscores for the meta-analyses as “zmeta” in the text and tables to distinguish the meta-analyses tests from other statistical tests. Prior to conducting the meta-analysis, we also generated a Q-statistic for all variables within each model. The purpose of the Q-statistic is to determine the extent to which the four years of data are likely to be drawn from one common population with regard to the particular variable being tested. A Qtest for network data was developed by Krackhardt & Kilduff (1999) to address problems associated with autocorrelated data. The resulting Q-statistic is asymptotically distributed as χ2 with (k -1) degrees of freedom. A high Qstatistic with a significant p-value suggests that the four years are not all drawn from the same population with regard to the particular variable. In other words, a significant Q-statistic suggests that all four years did not behave in a consistent way, thus making interpretation of the meta-analysis difficult. The Q-statistics (Q-scores) are reported in the tables along with the zmeta. Results Table 1 provides a summary of the composition of the sample during project 1 for each of the 4 years. As indicated in table 1, the mean group size ranged from 3.80 to 5.71 over the 4 years with 7 to 10 groups per class. Choosing Workgroup Members 24 Between 21 and 25 percent of the students were female with no significant difference in percentage female from year to year, χ2 (3, N = 165) = .285, p < .975. The percentage of non-white students varied significantly over the 4 years, χ2 (3, N = 165) = 9.495, p < .025. This is largely accounted for by the small percentage of minorities during the first year as compared to later years. Students' average GPA over the four years ranged from 2.39 in year 3 to 2.72 in year 1.